Systems and methods for integrating and using augmented reality technologies

ABSTRACT

The present disclosure generally relates to systems and methods for creating, publishing, accessing, and sharing AR, VR, and/or XR content. In embodiments, users may collaborate in an XR environment. In embodiments, a system disclosed herein includes a backend module and a user client that permits creation and/or viewing of XR content. Embodiments enable users to create customized XR content that is published to users based on predetermined times and/or locations. Embodiments provide for training and collaborative XR environments accessed by multiple users simultaneously.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Non-Provisional patentapplication Ser. No. 17/520,155, filed Nov. 5, 2021, and claims thebenefit of and priority from U.S. Provisional Patent Application Ser.No. 63/110,281, filed on Nov. 5, 2020, the contents of each of which arehereby fully incorporated herein by reference.

BACKGROUND OF THE DISCLOSURE

The present disclosure generally relates to extended reality systems andmethods of use thereof. More particularly, the present disclosurerelates to systems and methods that provide for the creation, authoring,consumption, distribution, display, use, and tracking of extendedreality content and experiences generally and augmented-reality contentand experiences specifically.

Extended reality (XR) is a general term referencing experiences createdfrom a combination of a real environment with virtual content, such asis created through human-machine interactions generated by computertechnology (including wearables). Examples of XR include (i) augmentedreality (AR), wherein a user's perception of objects that reside in thereal world is enhanced by computer-generated perceptual information;(ii) virtual reality (VR), wherein an environment is simulated for auser; (iii) mixed reality (MR), wherein real and virtual environmentsare “mixed” to produce new environments and visualizations permittingphysical and digital objects to co-exist and interact in real time; and(iv) hybrids, combinations, and variations of each of the foregoing. Assuch, XR encompasses everything from entirely “real” environments (whichmay be supplemented by “virtual” information) to entirely “virtual”environments (which may be based on or incorporate some “real”information).

Existing AR experiences typically involve enhancing real world objectswith “virtual” or computer-generated perceptual information, through oneor more of a user's senses (i.e., visual, auditory, haptic,somatosensory and olfactory). In addition, AR generally featuresreal-time interaction and accurate representation of both virtual andreal objects. This is typically accomplished constructively (by“overlaying” virtual information on top of a “real” environment) or“destructively” (by masking a portion of the real environment). Forexample, an AR experience may be provided through the use of gogglesthat are either constructive (superimposing additional virtualinformation on top of the user's perception of the real environment) ordestructive (obscuring portions of the real environment). In thismanner, the user experiences a seamless combination of the “real” and“virtual” environments. AR is largely synonymous with MR, although MRcan encompass fully virtual environments in which “real” objects arealso incorporated.

XR generally and AR specifically advantageously permit users toexperience a blended perception that integrates information immersivelywith the real environment around them.

Despite the obvious practical applications of XR generally and ARspecifically across a wide range of fields, existing technologies andsolutions suffer from a number of significant drawbacks.

Many existing AR solutions are platform-dependent, requiring contentthat is created for a specific user device. This makes it difficult towidely deploy AR content, as the content must be re-authored for eachseparate platform that will be used to experience the content. Users arefurther “locked-in” to content created solely for their existingplatforms, making transitions or upgrades more costly and timeconsuming, particularly where an existing AR content library must beadapted or “reauthored” for a new platform.

Content discovery is also lacking on existing platforms andtechnologies. Users must typically select specific content to display.This requires user education as to what applicable content may beavailable in different contexts and locations, creating a significantbarrier to user adoption and obviating some of the efficiency gains thatare provided by use of AR content through the increased time and effortrequired for users to find and obtain content relevant to particularcontexts or locations.

Authoring AR content for existing systems is also a laborious andcomplicated process, typically requiring expert manual involvement tocreate content for each specific platform. This raises the barrier toentry and increases the costs and time required for a user to createcustom AR content that may be relevant to that specific user.

Existing AR solutions are also not adaptable to incorporate newmodalities or enable use of MR, VR, or other XR experiences. This limitsthe scope of experiences available to users, and limits the types ofcontent that users can access without having to undertake complicatedand/or expensive transitions to new or different platforms in order toaccess new content.

Therefore, there is a long-felt but unresolved need in the art forimproved XR systems and methods generally, as well as improved ARsystems and method specifically, that address the foregoingdisadvantages as well as other disadvantages of existing technologies.

BRIEF SUMMARY OF THE DISCLOSURE

This summary is provided to introduce a selection of concepts in asimplified form that are further described in the detailed descriptionof the disclosure. This summary is not intended to identify key oressential inventive concepts of the claimed subject matter, nor is itintended for determining the scope of the claimed subject matter.

Generally, a system as disclosed herein may include a backend module anda user client that permits creation and/or viewing of XR content.

An embodiment provides a system comprising a server comprising adatabase, one or more first processors and a first memory comprisingfirst instructions, the server communicatively coupled to a user device;wherein the first instructions, when executed by the one or more firstprocessors, cause the server to perform operations comprising:receiving, from the user device, a first command to create new ARcontent; displaying, via the user device, a first plurality of optionseach corresponding to a type of AR content; receiving, from the userdevice, a second command specifying a first one of the first pluralityof options; generating an AR experience of the type corresponding to thefirst one of the plurality of options; and storing the AR experience inthe database.

A further embodiment provides a system comprising: a user device; aserver communicatively coupled to the user device and one or moredatabases, the server comprising one or more first processors and afirst memory comprising first instructions; wherein the firstinstructions, when executed by the one or more first processors, causethe server to perform operations comprising: transmitting, to the userdevice, a first set of commands configured to cause the user device togenerate a graphical user interface; receiving, from the user device, afirst set of information; transmitting, to the user device, a second setof information from the one or more databases configured to cause thegraphical user interface, to display an XR experience, wherein thesecond set of information is selected based at least in part on thefirst set of information; receiving, from the user device, a third setof information corresponding to one or more actions taken on the userdevices while viewing the XR experience; and based on the third set ofinformation, changing the contents of at least one of the one or moredatabases.

A further embodiment provides a method of displaying an XR experience,the method comprising the steps of: displaying, on a user device, aplurality of options each corresponding to an XR experience; receiving,from the user device, a selection corresponding to a first one of theplurality of options; displaying, on the user device, the XR experiencecorresponding to the selection, receiving, from the user device whiledisplaying the XR experience, first sensor data corresponding to aposition of the user device; receiving, from the user device whiledisplaying the XR experience, second sensor data corresponding to a realworld image proximate the user device; receiving, from the user devicewhile displaying the XR experience, third sensor data corresponding toinputs provided to the user device; and adjusting the displayed XRexperience based at least in part on the first sensor data, the secondsensor data, and the third sensor data.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

The foregoing summary, as well as the following detailed description ofthe disclosure, is better understood when read in conjunction with theappended drawings. For the purpose of illustrating the disclosure,exemplary constructions of the inventions of the disclosure are shown inthe drawings. However, the disclosure and the inventions herein are notlimited to the specific methods and instrumentalities disclosed herein.

FIG. 1 exemplarily illustrates a block diagram of a method of creatingAR content in accordance with an embodiment.

FIG. 2 exemplarily illustrates a block diagram of a method of preparingAR content in accordance with an embodiment.

FIG. 3 exemplarily illustrates a diagram of a system for tracking andreporting information in accordance with an embodiment.

FIG. 4 exemplarily illustrates a block diagram of a system in accordancewith an embodiment.

FIGS. 5 a-5 c exemplarily illustrate user interfaces for software fordisplaying AR content in accordance with an embodiment of the invention.

FIG. 6 exemplarily illustrates a block diagram of a method of creatingAR content in accordance with an embodiment.

FIG. 7 illustrates an exemplary process creation flow using the cloudeditor in accordance with an embodiment.

FIG. 8 an exemplary process flow for creating a geo-located ARexperience in accordance with an embodiment.

FIG. 9 an exemplary process flow for viewing a geo-located ARexperience.

FIG. 10 illustrates an exemplary user interface of an embodiment of thecloud dashboard in accordance with an embodiment.

FIG. 11 illustrates an exemplary user interface of the platform createmodule when creating a “base experience” in accordance with anembodiment.

FIG. 12 illustrates an exemplary user interface of the platform createmodule when creating an “advanced experience” in accordance with anembodiment.

FIG. 13 illustrates an exemplary user interface for choosing an ARlocation in accordance with an embodiment.

FIG. 14 illustrates an exemplary user interface for uploading assets inaccordance with an embodiment.

FIG. 15 illustrates an exemplary user interface for reviewing orpreviewing an AR in accordance with an embodiment.

FIG. 16 illustrates an exemplary user interface for choosing apublication time frame for an AR experience in accordance with anembodiment.

FIG. 17 illustrates an exemplary user interface for reviewing existingAR campaigns in accordance with an embodiment.

FIG. 18 illustrates an exemplary user interface of the “rooms” tab inaccordance with an embodiment.

FIG. 19 illustrates an exemplary user interface of the loading screenwhen viewing another user's room through a desktop client in accordancewith an embodiment.

FIG. 20 illustrates an exemplary user interface of the User App runningon a smartphone at the login screen in accordance with an embodiment.

FIG. 21 illustrates an exemplary user interface of the User App whileworking with an AR experience in a room in accordance with anembodiment.

FIG. 22 illustrates an exemplary user interface of the cloud creationeditor in accordance with an embodiment.

FIG. 23 illustrates the exemplary AR experience created in FIG. 22 inaccordance with an embodiment.

FIG. 24 illustrates an exemplary interaction with an AR object placed ina real world environment in accordance with an embodiment.

FIG. 25 illustrates a VII mode of operation in accordance with anembodiment.

FIG. 26 depicts an exemplary illustration of such variable content inaccordance with an embodiment.

FIG. 27 depicts an exemplary illustration of a user interface for groupcollaboration in accordance with an embodiment.

FIGS. 28 through 44 depicts exemplary illustrations of a user interfacefor a mobile application in accordance with an embodiment.

DETAILED DESCRIPTION OF THE DISCLOSURE

The following disclosure as a whole may be best understood by referenceto the provided detailed description when read in conjunction with theaccompanying drawings, drawing description, abstract, background, fieldof the disclosure, and associated headings. Identical reference numeralswhen found on different figures identify the same elements or afunctionally equivalent element. The elements listed in the abstract arenot referenced but nevertheless refer by association to the elements ofthe detailed description and associated disclosure.

Overview of the TeamworkAR™ System.

Embodiments of the present disclosure relate to improved XR systems andmethods termed the “TeamworkAR™ System.” As used herein, the term “AR”may encompass any form of XR (although AR is specifically contemplated)unless otherwise expressly noted. In exemplary embodiments, theTeamworkAR™ System comprises a hardware platform or device on which asoftware application (termed the “User App”) is run, a cloud dashboardpermitting configuration of the system, one or more experiences that maybe hosted remotely from the User App or incorporated locally therein, alearning record store (LRS) databases for online and offline trackingand reporting, and one or more communication channels (including video,audio, and/or text). The User App operates on a local device and mayinclude variation optimized for different platforms (such as iOS,Android, browser based, and wearables (such as the Realwear HMT-1 andMicrosoft HoloLens)). In an embodiment, content created for the User Appis platform agnostic, and capable of display and interaction with UserApps running on different devices. In embodiments, the capabilities ofthe User App are automatically enabled or disabled based on the featuresof the platform on which it runs, with the same content automaticallyadjusted based on the available capabilities.

FIG. 4 illustrates an exemplary embodiment of the TeamworkAR™ System. Asshown, one or more devices 302 (such as a headset 302 a, a tablet 302 b,a computer 302 c, a smart watch 302 d, and a smartphone 302 e) access acloud-based “thin” client 304. In an alternative embodiment, the devices302 instead utilize a locally run User App instead of the thin client304. Information in the thin client 304 or User App is synchronized withone or more databases 306. As shown, the databases 306 comprise a LRS306 a coupled to a separate learning asset host database 306 b, aLMS/LPX database 306 c, and a HRIS database 306 d. These databases 306allow information to be obtained from and shared with various outsidesystems.

Embodiments include a User App that is configurable to function acrossmultiple devices (including smartphones, laptops, tablets, wearables,and other computing technologies). In embodiments, the User App providesusers thereof with access to self-guided assistance in an AR setting,providing utility in a wide variety of fields including entertainment,technical design and engineering, help-desk settings, and providinggeneral training and support. The User App is configured to work inwidely disparate settings and fields based on the specific contentprovided through the User App. In an embodiment, the User App isconfigured to run on a standard web browser. In an embodiment, the UserApp is configured to automatically enable and disable capabilities basedon the specific sensors, displays, and other hardware features of thedevice on which the User App is run.

Embodiments include authoring software which allows operators to create,customize, and activate XR experiences. These XR experiences may beautomatically customized by the User App depending on whether it is runfrom the web, a mobile device, or a more capable hardware platform. Inembodiments, backend software is provided that includes an applicationprogramming interface (API) that is accessible from User Apps running inboth web clients as well as on other platforms.

In an embodiment, users of the User App may be placed in groups (termedorganizations), with each organization able to separately subscribe toXR content experiences. In an embodiment, authoring software employsrobot process automation to transform large collections of pre-existingcontent into AR or XR experiences. This includes scripted photogrammetryconversion, object transcoding and conversion, and deployment acrossmultiple channels, such as web-based or through a mobile app.

In embodiments, systems employ a variety of AR technologies,geo-location placement, computer vision, feature-point collection,feature-point retrieval, and learning records stores (LRS) in accordancewith the xPAI eLearning specification to empower individuals to interactwith artifacts in an AR experience and be able to validate the learningactions of the user on the AR artifact, and be able to report thoseactives to a data reporting repository.

Embodiments provide users with the ability to create AR objects andimages from various forms of graphics text, and video, including JPEG,MP4, and 3D models, and place them via location identifiers into reallocations. Embodiments permit the bulk ingestion of data, which are thenautomatically converted and placed as AR content that is geo-tagged to acorresponding real word location. By tracking a user's real worldlocation, appropriate AR content may then be displayed to the userthrough the User App.

Cloud Dashboard

Embodiments provide a web-based cloud platform through which users maymanage content, experiences, and create new AR content for distribution.In an embodiment, the cloud dashboard enables users to upload media(including, pictures, video. and 3D models), convert the media to XRand/or AR content, and distribute such content to members of theirorganization. In an embodiment, the cloud dashboard enables users tosearch for and locate new XR and/or AR content based on specificgeo-locations. In an embodiment, the cloud dashboard permits users todiscover new XR and/or AR content via point cloud database (such as avirtual digiboard, navigation wayfinder). In an embodiment, the clouddashboard enables users to engage in web-based teleconferencing withother users through XR and/or AR tools. In an embodiment, the clouddashboard enables users to store and retrieve video experiences to andfrom the cloud platform. In an embodiment, the cloud dashboard enablesusers to invite others to connect (e.g., by email or direct accountassignment). In an embodiment, the cloud dashboard enables users tointegrate workflow into popular IT service management (ITSM) softwarelike ServiceNow™ In an embodiment, the cloud dashboard enables users toget detailed reports via Tableau and other rich reporting tools.

In an embodiment, the cloud dashboard enables users to employ existingcontent management system (CMS) tooling to manage users, roles, andorganization structures. In an embodiment, the cloud dashboard enablesdesignated users to push certain XR and/or AR content to other membersof their organization.

In embodiments, the features of the cloud dashboard may also be providedlocally (such as through the User App).

FIG. 10 illustrates an exemplary user interface 1000 of an embodiment ofthe cloud dashboard in accordance with an embodiment of the presentdisclosure. As shown, users are presented with options to “join ameeting” 1002 or “host a meeting” 1004 (with such meeting enabled withXR and/or AR capabilities as discussed herein). Users can furtherexplore active “rooms” or XR and/or AR experiences currently viewed byother users 1006, manage users within their organization and/or modelsavailable to the user 1010, manage media (such as video) available tothe user 1012, manage the organizations to which the user is a member1014, view usage logs 1016, manage existing AR experiences to which theuser has access 1008, and create new AR experiences 1018.

FIG. 18 illustrates an exemplary user interface 1800 of the “rooms” tab.As shown, users may either join 1802 a personal “room” (wherein adesired AR experience may be displayed) or invite 1804 others to jointheir room. Invitations can be sent through the cloud dashboard or byproviding another user with a direct link 1806.

FIG. 19 . illustrates an exemplary user interface 1900 of the loadingscreen when viewing another user's room through a desktop client. Asshown, the user joining the room may choose to give the software accessto the camera 1902 on his or her local device in order to enable videoparticipation and/or the display of AR content that incorporates aspectsof the user's real world environment. An alternative embodiment of thisuser interface on a smartphone is illustrated in FIG. 5 b.

FIGS. 5 a through 5 c illustrate a user interface 500 on a smart phone.As shown in FIG. 5 a , while in a room, all users may view the same realworld environment 502 and create AR annotations 504 visible to the otherusers. Users may simultaneously communicate via audio, video, and/ortext; video feeds 506 b and/or user icons 506 a are displayedsimultaneously with the rest of the interface. As shown in FIG. 5 b , auser may be prompted to give camera access 508 in order to share videowith other users. As shown in FIG. 5 c , addition of AR content is notrequired, as users can instead choose to share only the real environment(or only audio, video, and/or textual information). As well, users canmove augmented reality content (scale, rotate, and translate) togetherin real time via a media synchronization system built into all themobile and web applications.

Creation of AR Content.

In the embodiment shown, content creation is controlled by a platformcreate module and a cloud editor. In an embodiment, the types of contentthat may be created by the platform create module are categorized aseither “base experiences” (or “basic experiences”) or “advancedexperiences.” The cloud editor allows animated free-form creation ofbespoke AR content to be created and deployed.

FIG. 1 depicts an exemplary block diagram of a method 014 of creating ARcontent. As shown, the method starts at step 000. At step 001, the userlogs into system and selects a “create” option. The user then selects toeither create a simple (or basic) AR event 002, create a complex (oradvanced) AR event 003, or create a free-form AR event 004. If the userselects a simple event 002 or complex event 003, the user then ispresented with predetermined options for the type of desired content005, 006 before proceeding to the creation flows 008, 010 (as shown onFIG. 2 ). Each of these options is discussed in greater detail below.For free-form AR events, the user is taken to the cloud editor 007 (asdiscussed below), whereupon a determination is made as to whether thecontent is to be app based 011 or online-based, before ultimatelypublishing the newly created content to local storage 012 and/or aweb-based format 013.

FIG. 11 illustrates an exemplary user interface 1100 of the platformcreate module when creating a “base experience” 1102. Basic experiencesare simple, modular AR experiences like recognizing an image or itemwith a camera and presenting an AR experience based on recognition ofthe image. In the embodiment shown in FIG. 11 , the base experiencesavailable are an image-recognition event (i.e., recognizing an imageobtained from a camera and/or otherwise uploaded to the system) 1104;creating a model placement (i.e., placing an existing model in an ARexperience) 1106; placing an instruction guide (i.e., placing anexisting guide in an AR experience) 1108; creating a virtual award 1110;creating a virtual video message (i.e., a recorded video that mayinclude AR content) 1112; and creating an AR location experience (i.e.,an AR experience that is triggered when a user is at a physical, realworld location) 1114. In an embodiment, the system takes simplemultimedia (such as pictures, videos and 3D models) and packages them asdiscoverable AR experiences. These experiences can be authored toprovide “call to action” buttons and are trackable. As shown, users canaccess a top-level menu 1118 with option to view rooms, people,invitations, AR experiences, settings, user management, modelmanagement, video management, and organization management. Further,users have the option to create view current campaigns 1120.

FIG. 2 depicts an exemplary high level process creation flow foringested content. The method begins by ingesting the base model at step200. Next, one or more RPA scripts are called at step 201. At step 202,media conversion is effectuated through the use of one or more tools andscripts. At step 203, content is stored in an applicable format. At step204, the web API is notified that the process has completed.

FIG. 6 depicts a specific exemplary process creation flow 600 foringested content. As shown, the method begins at step 602. At step 604,the server sends a start job request to an “orchestrator” process aftera user uploads a file to convert to AR content. At step 604, theorchestrator module begins one or more conversion job(s) with the inputfrom the server (such as file name, file location, etc.). At step 606,the RPA job commences with input from the orchestrator. At step 608, therobot receives the job request before the RPA job downloads requestedfile from S3 at step 610. At step 614, a decision is made whether toconvert using Vectary (e.g., checking whether Vectary is available); ifso, the conversion occurs at step 626 before the robot uploads theconverted file(s) at step 620. Otherwise, a decision is made at step 616whether to convert using Blender (e.g., checking whether Blender isavailable); if so, the conversion occurs at step 628 before the robotuploads the converted file(s) at step 620. If Blender is not available,the conversion occurs using Maya at step 618 before the robot uploadsthe converted file(s) at step 620. As will be clear to one of skill inthe art, other third-party model software may be used in place of thoseindicated and additional (or fewer) checks may be made. In eachinstance, the conversion step 618, 626, 628 involves importing thefile(s), selects the appropriate file type to convert to, and exportingthe file(s) (as appropriate). At step 622, the RPA job notifies thesystem that the conversion is finished. A check is performed at step 624for any additional jobs, and if none, the method ends at step 630.

FIG. 12 illustrates an exemplary user interface 1200 of the platformcreate module when creating an “advanced experience” 1202. Advancedexperiences are compound experiences that may be tailored for moreparticular user needs. In the embodiment shown, the advanced experiencesinclude creating a chatbot 1204; creating a scavenger hunt 1206;creating a map waypoint system 1208; requesting a robotic processautomation (RPA) scrape (i.e., using RPA automation to obtaininformation, such as from one or more websites) 1210; and creating acustom animation (which invokes the cloud editor) 1212.

In an embodiment, AR content and experiences are authored in adevice-agnostic way. This permits the AR content to be accessed on anydevices capable of running the User App. Location information (includingGPS and elevation) may be captured using whatever sensors are providedon the device. For users on devices lacking appropriate sensors,information can be provided manually (such as by picking or setting analtitude, including through providing information such as “I am on the34th floor of Comcast Center”). Automated tools can the availableinformation and automatically translate it to the desired geolocationdata, or involve a human operator for assistance with the neededtranslation. Where further sensors are available (such as a barometeraltitude), this can be used directly to pinpoint the user's location.

Through this approach, users may precisely associate arbitrary ARexperiences with specific real world locations. This enables thecreation of subscription services that advertise to others or allowspecificity/time-bound experiences. Further, content may be deliveredwith specificity, providing better integration into placement zones(e.g., multiple items, physical location assets, temperature of lights).In an embodiment, AR content and experiences are coded with geographicinformation (such as latitude, longitude, and/or altitude information);this geographic information is then used by the cloud dashboard and/orUser App to associate the AR content and experiences with a desired realworld location. In an embodiment, AR content and experiences aretriggered when the device running the cloud dashboard and/or User Appapproaches the real world location identified by the geographicinformation and the AR content and experience is displayed only at thereal world location. In an embodiment, each instance of the clouddashboard and/or User App is able to communicate with every otherinstance within a certain proximity, enabling all such instances toshare event location and information in real spaces such thatinteractions between instances are enabled, allowing for a commonexperience across instances.

In an embodiment, the cloud dashboard and/or User App is capable ofauthoring AR experiences (including specific placement, type, andcontent) from any device configured to either the cloud dashboard or runthe User App. In embodiments, AR experiences are authored using openstandards (such as GeoPose, Machine Readable World, and SpatialDiscovery Service).

In an embodiment, the cloud dashboard and/or User App is configured towork with ARKit and ARCore. Content creators may access multiple valuesvia light estimation, including a light intensity value and an averagecolor tone value. In an embodiment, post-processing effects in Unity 3Dare used to improve placement and visual readability of AR content andexperiences.

The following describes an exemplary process in accordance with anembodiment for creating a location-based AR experience. Suchlocation-based experiences may be discoverable only to users physicallylocated in the specified location. Locations may be specified withgranular detail, including through use of longitude, latitude, andaltitude. In embodiments, locations may be specified through othergeo-location means, such as requiring a local device to be connected toa particular wired or wireless network, or to receive a signal from atransmitter (such as an RFID tag).

As shown in FIG. 13 , a user first chooses an AR location 1102 by usinga map 1104, entering an address 1106 (which, in the embodiment show,optionally includes a floor 1110 of a building), or physicallytravelling to the location before accessing the User App or the clouddashboard. The user may define a specific radius 1108 within which theAR location will apply. The user interface includes indicators 1112 1112indicating the progress made towards deploying or “going live” with theexperience.

As shown in FIG. 14 , the user next uploads new assets 1402 and/orselects previously provided assets 1106 for the experience. In theembodiment shown, assets are uploaded by dragging and dropping them on aspecified area. As will be clear to one of skill in the art, alternativeapproaches to this user interface may be used (such as opening a filebrowser).

As shown in FIG. 15 , the user can then preview the experience. In theembodiment shown, the user can scan the presented QR code 1502 using alocal device running the User App and having a camera or scanner toautomatically engage the experience without having to physically travelto the location associated with the experience. This permits userslocated remotely from a physical location to create and test experiencesfor that location.

Once the experience is completed and no further changes are needed, asshown in FIG. 16 , a user can then specify a name 1062 and choose a timeperiod 1604 over which the experience will be available. The user maypublish the experience 1606 or choose to save the experience for later1608 (without publishing it to other users). This permits users tolaunch time-and-location-based campaigns that automatically expire andare only available to users at specific, real world locations.

As shown in FIG. 17 , a user can review all active campaigns (i.e.,experiences currently published to other users) 1702, inactive campaigns(i.e., experiences no longer published) 1704 and revise 1706 or delete1708 them as needed. In an embodiment, campaigns can be published ordeactivated with a single toggle 1710. Details 1712 regarding eachcampaign are provided, including the type, status, number of views,dates active, and publication status. New AR experiences can also becreated 1714 as discussed here.

FIG. 8 depicts an exemplary process flow 800 for creating a geo-locatedAR experience. The process begins at step 802 as the user logs in andcreates an AR experience at step 804. The user then chooses ageolocation (as discussed above) for the experience at step 002. Theuser can customize the precise location (in 3D space) in which eachelement of the model will appear before selecting a start and end date(if desired) for the experience to be available.

FIG. 9 depicts an exemplary process flow 900 for viewing a geo-locatedAR experience. The user begins at step 902 by logging in. The systemthen obtains information for nearby objects and/or AR experiences atstep 904. A user can choose to either view the nearest experience (orany experiences of which the user is in range) at step 906 or select oneor more experiences from a menu at step 908. At step 910, the selectedexperiences are either accessed (if already locally cached) ordownloaded and placed in virtual space around the user. The user canthen view the experience at step 912 by moving the user's device todisplay various portions of the real and virtual environments.

FIG. 20 illustrates an exemplary user interface 2000 of the User Apprunning on a smartphone at the login screen. In the embodiment shown, inorder to access the User App, the user must enter his or her credentials(shown as comprising a username 2002 and password 2004, although othercredentials may be used) which are then confirmed via a credentialdatabase stored either locally or remotely.

FIG. 21 illustrates an exemplary user interface 2100 of the User Appwhile working with an AR experience in a room. A shown, AR content 2100(here, annotations shown in dashed lines) are shown directing the useras to how to interact with the real world (i.e., by removing a specificwire). The user can leave the room by “hanging up” and has the option todisable audio and/or video sharing with another member of the room usinga menu 2104. Users can create AR annotations using the tools 2106provided at the top of the screen, with those annotations then visibleto other users in the same room. In the embodiment shown, the roomdisplays the real environment of one user through the camera on thatuser's device to all other users in the room. Those other users can thenprovide AR annotations while also communicating via audio, video, and/ortext. This enables remotely located users to guide one another throughcomplex, real world operations.

FIG. 22 illustrates an exemplary user interface 2200 of the cloudcreation editor. As shown, a user may place one or more objects 2202which can then be animated. Text 2204 may be added to the objects ordisplayed as annotations in the AR experience. Each virtual item may beseparately manipulated and/or animated using the provided tools 2206.New behaviors 2208 and actions 2210 may be created, and the animationsmay be previewed 2212

FIG. 23 illustrates the exemplary AR experience 2300 created in FIG. 22. As shown, text 2304 c and images 2304 a, 2304 b are configured to besuperimposed over a real world environment 2302 at a specific real worldlocation. Users can then interact with the AR content (such as bycalling or emailing the indicated real estate agent) using hyperlinksincluded in the AR content. Similarly, hyperlinks in the AR contentpermit the user to obtain more information (such as by accessing awebsite or engaging in a more elaborate AR experience, such as bytouring a property in a virtual environment).

FIG. 7 illustrates an exemplary process creation flow 700 using thecloud editor. As shown, users begin 702 by opening the cloud editor 704and logging in with their credentials 706 before choosing to either loadan existing scene or create a new scene ate step 708. At step 711, theuser drags one or more user interface icons into the workspace to createone or more objects. At step 712, the user edits object properties bydragging the object around the 3D environment and selecting one or moreproperties from the menu. Alternatively or concurrently, at step 710,the user may create one or more behaviors before editing the behavior ofthe object using the provided menu at step 714. At step 716, the usercan select to save the scene for later use or publication beforeuploading the scene at step 718.

FIG. 24 illustrates an exemplary user interface 2400 for an interactionwith an AR object 2402 placed in a real world environment 2404. Asshown, the user can fully interact with the AR object 2402 usingcontrols 2506 provided by the User App. This enables users to performtasks such as pre-work or pre-study of working using the User App beforehandling the physical object itself. For example, a user may test outspecific procedures using mobile gestures (providing a simpler form ofhands-on training) or obtain remediation or refresher work as neededwithout needing to have access to the physical object. This isparticularly advantageous for users in a factory setting who may not beable to interact with physical machinery without stopping factoryoperations (potentially costing significant time and/or expense in lostproduction).

FIG. 25 illustrates a VII mode of operation 2500. In the embodimentshown, a user may interact with a virtual representation of a real worldobject in a virtual environment. Such VR experiences can incorporateexisting VR technology, such as the Oculus Quest, and allows for anentire real world task to be simulated. This enables users to obtainimmersive training (which may be necessary for formal certification)without requiring access to a physical object. In embodiments, guidanceis provided in the virtual environment (such as instructions forcompleting the task) along with real-time feedback based on the user'sactions. In an embodiment, an instructor can remotely monitorperformance and provide real-time feedback. In the embodiment shown, avirtual representation of a machine 2502 is displayed. The user isdirected to simulate placing a part 2504 a onto a specific location 2506of the machine 2502 using VR annotations 2506; the user must match therepresentation of the part 2504 a with the VR annotated version thereof2504 b.

In an embodiment, the User App provides a feedback mechanism whereby allskill-related user actions detected by the system are recorded to an LRSvia xAPI statements. In an embodiment, such actions are sent to anonline performance tracking (e.g., Twilio) via natural languageunderstanding (NLU) functions. Embodiments provide a chatbot configuredto provide interactive voice response (IVR) to users to provideimmediate interactive feedback. Specific feedback may includenotification of an “automatic failure” or action performed virtuallythat would have been dangerous if performed in a real environment,notification of excellent or standard proficiency with a particulartask, instant remediation of errors, and interventions based on poorperformance. In an embodiment, tracking statements are weighted, andoverall pass/fail remediation thresholds are either preconfigured by anadministrator or weighted by pre-defined importance (i.e., breakage,danger).

For embodiments employing online tracking, in an embodiment, an offlinedevice is configured to store tracking information in offline storageand upload all locally cached records to online storage once a networkconnection is detected.

Use of Robot Process Automation (RPA)

Embodiments employ RPA to enable data intake and processing from publicand/or private databases. In an embodiment, AR content may be createdand populated based on RPA processing of existing, non-AR content. In anembodiment, RPA provides mass ingestion of pre-existing non-AR content,creates AR content, and associates the newly created AR experiences withother AR content and/or real world locations. Existing information maybe obtained by utilizing a screen scrape, OCR process, and/or otherautomated method before performing processing to match obtained contentto pre-created AR templates. Such custom AR content can beauto-populated with features including widgets, interactivity, calls toaction, web links, and the like. AR content may be populated via acombination of RPA and JavaScript Object Notation (JSON) manifestsbefore being combined into single packages (e.g., in the ZIP format).Once obtained, AR content may be manually or automatically deployed tospecific users or organizations. Users may be provided with notificationindicating the availability of newly created content.

In an embodiment, content is automatically combined into AR experiencesincorporating information at varying distances (such as far, closer, andnear) wherein the displayed content is adapted based on the user'sdistance from a location. FIG. 26 depicts an exemplary illustration ofsuch variable content. As a user approaches a location, distant content2602 may be displayed providing high-level information relevant to thelocation. As shown, this high-level information may include informationas to whether a property is “for sale” and, if so, the current askingprice. As the user approaches, the display of this information may adaptand be supplemented to “closer” content 2600. Here, the “closer” contentcomprises a virtual “for sale” sign and one or more images from insidethe property. If the user desires, “close” content 2604 is thendisplayed. Here, the “close” content comprises a floor plan of aspecific property. As shown in 2606, this content may be generated byusing RPA to obtain information from a website, process it into relevantAR content, and populating the AR experience at a real world location.

FIG. 3 illustrates an exemplary method of processing content using RPAto create AR content. As shown, the method begins at step 100 as one ormore items are loaded into the system. At step 001, the system processesor ingests the provided content and categories the content as being apicture 102, model 103, or video 104. “Other” content may be manuallymatched to an existing template (such as, for example, textualinformation that should be displayed graphically). For each type ofcontent, the system proceeds to prepare it 105, 106, 107 beforeassigning it to an applicable geo location at step 108. A user is thenprovided an opportunity to preview or review the generated content atstep 109 before the content is published (either to app-based storage oran online database).

Overview of the TeamworkAR™ System.

In an embodiment, the TeamworkAR™ System includes a reporting mechanism,enabling the recordation of user's skill competencies via AR testprocedures. Specific AR experiences are presented to a user, such as torepair a device. Sensors on device are used to track the user's realworld operations (e.g., by using a camera with image recognitionenabling the system to track the user's real world movements, includinghand placement and operations). The system can thus provide adaptiveexperience where mistakes or errors made by a user are addressed throughinstantaneous AR feedback. In extreme cases, the user may be forced toreset and/or restart the operation until it is performed correctly. Inthis manner, the system may provide safety features where anypotentially unsafe procedures are identified and the user is alertedbefore proceeding, allowing for the user (and/or others in the user'sorganization) to receive real-time feedback on the user's performance.

In an embodiment, the Unity XR Toolkit is used to create experiences.Embodiments provide functionality such as eye-tracking in addition totracking of macro and/or micro movements by a user. In an embodiment,this compliance tracking is used to provide certification for tooland/or hand operations.

Exemplary Uses of Existing Technologies.

Embodiments are configured to utilize existing AR-adjacent technologiesto provide a seamless AR experience. In an embodiment, eye-trackingimplementations are accomplished through integration with AR Foundation,the Unity XR Toolkit, and underlying XR plugins available on Unity3D. Inan embodiment, Twilio Autopilot is used to provide NLU functionality. Inan embodiment, optical character recognition (OCR) and OpenCV is parsedand performance indicators sent to an IVR-capable chatbot to enableunderstanding and responses to text visible in a real environment. In anembodiment, performance indicators are stored as raw xAPI trackingstatements.

Exemplary Use Cases for the TeamworkAR™ System.

The following description provides exemplary use cases for the User App.These descriptions are illustrative (and not limiting) of the widevariety of fields of use in which embodiments of the present disclosuremay be advantageously employed.

In an embodiment, the User App is configured to assist withtrouble-shooting hardware issues. In this embodiment, the computingdevice on which the User App operates (such as a standard smartphone)includes a camera and a display. A user activates the User App such thatits user interface is displayed on the screen of the smartphone andpoints the camera of the smartphone at a device in need of repair. TheUser App, taking information obtained through the camera, compares theappearance of the device to a known database of devices that may berepaired in order to identify the specific device at issue. The User Appthen uses the user interface to display a series of pre-defined optionsto the user, identifying the types of repairs that may be performed.Once the user selects the specific repair task to be accomplished usingthe standard inputs on the smartphone (e.g., the touch screen includedin the smartphone's display), the User App presents the steps to becarried out in the repair using the user interface. The steps may bedisplayed as textual information and/or annotations on the device to berepaired. For example, if a series of screws must be removed, the userinterface may display text informing the user of the task to beaccomplished while also highlighting or otherwise directing the user'sattention to the specific screws to be removed. In the event that thedevice must be manipulated (e.g., turned in order to expose the screwsto be removed), the user interface may display annotations (such asarrows) or animations (such as a superimposed image of the devicerotating) in order to guide the user to perform the requisite action. Bymonitoring the user's process using the smartphone's sensors (e.g., thecamera), the User App recognizes once the problem has been resolved andis able to display the correct series of steps at the requisite time(and in the needed order) to accomplish the repair task.

Content for each procedure to be accomplished may be authored inadvance, as discussed herein, by reducing the procedure to a series ofrequired and alternative steps, possible workflows, and visuallyanimated representations.

By presenting information in this manner and adapting to the actionsperformed by a user, embodiments provide reduced costs, increasedefficiency, reduced downtime, enhanced training options, lower returnvolumes, and accelerated technology adoption.

In an embodiment, an operator equipped with a device running the UserApp in a factory is guided, with or without real-time human assistance,through a maintenance procedure. As before, each step in the maintenanceprocedure is displayed to the operator on the device using AR and thedisplay is updated as sensors on the device monitor the operator'sprogress.

In an embodiment, the User App is used in a training environment, wherea particular item is presented virtually to the trainee using the UserApp on a device. The trainee may interact with the item virtually,enabling the trainee to gain “hands on” experience repairing or usingthe item without the costs associated with obtaining, shipping, andotherwise providing the training with the real world item. Thiscompanies to easily and inexpensively scale to a large work group, astraining occurs completely in a digital environment.

In an embodiment, sales teams employ the User App salespersons tocomplement phone calls, emails, documents, and linear video with virtualdemonstrations that provide actual or prospective customers with ARpresentations to enrich the sales experience.

In an embodiment, the User App is used to provide customer support.Individuals needing support first access the User App, which attempts tomatch the individual's issue to the pre-loaded solutions contained inthe User App. If a match is found, the individual uses the User App toaddress the problem without requiring further involvement from customersupport. If a match cannot be found, the User App connects theindividual to a customer support agent using video, audio, and/or otherinformation provided by sensors on the individual's device running theUser App to allow for comprehensive troubleshooting and support. Thecustomer support agent can not only gather information from theindividual's device, but also display AR content to the user to guidethe user through a troubleshooting or repair process.

FIG. 27 depicts an exemplary illustration of a user interface 2700 forgroup collaboration in accordance with an embodiment. As shown, the userinterface 2700 may display video for a participant 2702, a stillphotograph for a participant 2706, or a textual indicator for aparticipant 2704.

Multiple individuals may be connected together and interact whilesimultaneously experiencing the same content 2718, which may be AR, VR,or audio-visual content. As shown, the content 2718 is a live videostream taken by the camera of one of the users; AR content may besuperimposed thereon or incorporated therein. The user interface 2700further comprises options to share video 2708, mute a user 2710, connectto speakers or another audio device 2712, access additional options2714, or end the connection 2716.

Exemplary Mobile Application

FIGS. 28 through 44 depicts exemplary illustrations of a user interfacefor a mobile application in accordance with an embodiment.

As shown in FIGS. 28 through 30 , in an embodiment a login screen 2800is displayed upon launch. The login screen 2800 may contain anidentifier 2802 of the application and/or its provider, a user field2804 to input a user identifier (e.g., a user name or email address), apassword field 2804, and a submit button 2808. In an embodiment,information is input using an on-screen keyboard 2810. In alternativeembodiments, the on-screen keyboard 2810 is omitted and information isinput using other means, such as by way of illustration and notlimitation, a physical keyboard, voice to text, or other input devicesas will be appreciated by one of skill in the art.

As shown in FIGS. 31 through 33 , in an embodiment a welcome screen 3100is displayed after authentication upon first launch. The welcome screen3100 may contain an identifier 2802 of the application and/or itsprovider, an info graphic 3102, 3112, 3116 illustrating uses of theapplication, informational text 3104, 3114, 3118 explaining uses of theapp, the option 3106 to access an information video or otherinformation, an indicator 3108 of position of the current view inrelation to the welcome screen 3100 as a whole, and an option 3110 toskip the introduction.

FIGS. 34 and 35 depict an embodiment of a user interface 3400 forselecting an AR experience. As shown, the user interface 3400 includesoptions to select AR experiences 3402 and rooms 3404, along with anoption to add an AR experience 3406. As shown in FIG. 34 , initiallythere may be no AR experiences available, and the user interface 3400displays a message 3408 to that effect. Once AR experiences are added,visual indicia 3502 for each available AR experience are displayed andmay be selected.

FIG. 35 depicts an embodiment of a user interface 3600 for downloadingassets. AR experiences and other content may be downloaded remotely foruse on a local application.

FIGS. 36 and 37 depict an embodiment of a user interface 3600 forselecting a room. As shown, the user interface 3600 includes options toselect AR experiences 3402 and rooms 3404, along with an option tocreate a room 3702. As shown in FIG. 36 , if no public rooms areavailable a user is provided with the option to either create a room3702 or join a specified private room 3704. Alternatively, if publicrooms are available, as shown in FIG. 37 a list 3706 of available roomsis displayed. The list may include details regarding the rooms, such asthe number of users in each, a topic or title for the room, or thename(s) of those already present in the room.

FIGS. 39 and 40 depict an embodiment of a user interface 3900 foraccessing a room. As shown, content 3912, which may be AR, VR, oraudio-visual content, is displayed to all users in a room. A user ispresented with options 3902 for reviewing the content, including toannotate the content 3912 or incorporate AR elements therein. As shown,the content 3912 is a live video stream taken by the camera of one ofthe users; AR content may be superimposed thereon or incorporatedtherein. Users further have the option to chat 3904 with other users inthe room, display a whiteboard 3906 to other users in the room, andtransfer files 3908 to users in the room. Controls 3910 enable a user tomodify the content 3912 (e.g., by flipping the camera), control audiooptions as discussed herein, leave the room, or invite users to theroom. As shown in FIG. 40 , the options 3902 are context sensitive andmay permit a user to annotate the content 3912 or add AR elements to thecontent 3912.

FIGS. 41 and 42 depict a user interface 4000 for conducting a chatsession with other users. As shown, users have the option 4206 to returnto a menu displaying available chats. The users included in the chat aredisplayed in a list 4002 above a series of messages 4004 sent in thechat. Users can input new messages in the input field 4008. In anembodiment, information is input using an on-screen keyboard 4010. Inalternative embodiments, the on-screen keyboard 4010 is omitted andinformation is input using other means, such as by way of illustrationand not limitation, a physical keyboard, voice to text, or other inputdevices as will be appreciated by one of skill in the art.

FIGS. 43 and 44 depict a user interface 4200 for adding users to a chat.Users can be added from the system 4210 (e.g., other users on the mobileapplication using separate devices and/or users accessing the systemthrough web browsers or other devices), via an SMS message 4212, and viaemail 4214. Once a user is invited, they receive a message with a linkto join the chat using the selected service. User can be located throughsearch 4208, and a list of available users 4206 on the selected serviceis displayed. One or more users may be selected 4216 and then addedautomatically or after opting in (e.g., by selecting the link).

Additional Embodiments

In an embodiment, the User App is configured to permit crowdsourcedsolution. As users solve new problems using the User App that are notaddressed by existing solutions stored within the User App, the User Appcan automatically record the steps taken by a user to either create newsolutions (that are then pushed out to other users) or to flagparticular problems as requiring development of new solutions.

In an embodiment, the User App is used to integrate AR, digitaltwinning, the Experience API (or xAPI) and computer vision to assistusers in learning on equipment that is not physically present, using ARto display the equipment while reporting the learning activities.

In an embodiment, the User App is used to provide field services. TheUser App provides step-by-step instructions, using AR, to customers,and/or technicians to allow them to complete a task or a repair. Theactions of the work individuals are tracked, and verified using computervison or other sensing technology, with immediate feedback provided andall activities recorded for learning and/or compliance.

Embodiments may be used in a variety of industries, includingpharmaceuticals, medical, and health care. Embodiments use geo-locationand anchor improvements to reduce drift along with computer vision forAR placement so as to align medical images on patients.

Embodiments provide AR digital instruction, video and audiocollaboration, drawing annotation, form scanning and image and objectrecognition.

Embodiments for in the oil, gas, utility, and mining industries provideVII safety simulation, oil rig virtual collaborations, self-directedSOPs and checklists.

Embodiments for the automotive, aviation, and aerospace industriesprovide digital twin, voice-enabled AR for customization, productwalkthroughs, and simulation.

Embodiments for the finance, real estate, and insurance industriesprovide for the bulk importation of sign and housing data and media.Location-based notifications may advise users that AR content ispresent. Existing data can be imported through form scans and ICR/OCRrecognition of form completion.

Embodiments for the manufacturing and heavy machinery industries providehands free “see what I see” video collaboration and archival.Embodiments provide digital twining of real objects for pre-training.Embodiments provide object recognition and steps with computer vision.Embodiments provide for offline distribution of AR content forlow-bandwidth/weak Internet applications.

Embodiments for use in the architecture, engineering, and constructionindustries provide point cloud manipulation to place AR elements withinscanned environments. LIDAR recording allows for recreation of scannedarchitectural sites and 3D engineering models generated from AR scans.

Embodiments for the travel and hospitality industries provide for 360camera walkthroughs of real locations in AR, including AR placement ofelements, walkthrough paths with destination maps, and virtual greeterswith an IVR chatbot.

The foregoing examples have been provided merely for the purpose ofexplanation, and are in no way to be construed as limiting of thepresent invention disclosed herein. While the invention has beendescribed with reference to various embodiments, it is understood thatthe words, which have been used herein, are words of description andillustration, rather than words of limitation. Further, although theinvention has been described herein with reference to particular means,materials, and embodiments, the invention is not intended to be limitedto the particulars disclosed herein; rather, the invention extends toall functionally equivalent structures, methods and uses, such as arewithin the scope of the appended claims. Those skilled in the art,having the benefit of the teachings of this specification, may affectnumerous modifications thereto and changes may be made without departingfrom the scope and spirit of the invention in its aspects.

Any other undisclosed or incidental details of the construction orcomposition of the various elements of the disclosed embodiment of thepresent invention are not believed to be critical to the achievement ofthe advantages of the present invention, so long as the elements possessthe attributes needed for them to perform as disclosed. Certainly, oneskilled in the XR field would be able to conceive of a wide variety ofalternative system configurations and successful combinations thereof.The selection of these and other details of construction are believed tobe well within the ability of one of even rudimental skills in thisarea, in view of the present disclosure. Illustrative embodiments of thepresent invention have been described in considerable detail for thepurpose of disclosing a practical, operative structure whereby theinvention may be practiced advantageously. The designs described hereinare intended to be exemplary only. The novel characteristics of theinvention may be incorporated in other structural forms withoutdeparting from the spirit and scope of the invention. The inventionencompasses embodiments both comprising and consisting of the elementsdescribed with reference to the illustrative embodiments. All technicalterms shall take on their customary meaning as established by theappropriate technical discipline utilized by those normally skilled inthat particular art area.

1. A system comprising: a user device; a server communicatively coupledto the user device and one or more databases, the server comprising oneor more first processors and a first memory comprising firstinstructions; wherein the first instructions, when executed by the oneor more first processors, cause the server to perform operationscomprising: transmitting, to the user device, a first set of commandsconfigured to cause the user device to generate a graphical userinterface; receiving, from the user device, a first set of information;transmitting, to the user device, a second set of information from theone or more databases, wherein the second set of information isconfigured to cause the graphical user interface to display an XRexperience, wherein the second set of information is selected based atleast in part on the first set of information; receiving, from the userdevice, a third set of information corresponding to one or more actionstaken on the user device while viewing the XR experience; based on thethird set of information, transmitting, to the user device, a fourth setof information from the one or more databases providing a response tothe one or more actions taken on the user device while viewing the XRexperience.
 2. The system of claim 1, wherein the first set ofinformation comprises a geographic identifier signifying a real worldlocation and wherein the XR experience corresponds to the real worldlocation.
 3. The system of claim 2, wherein the first set of informationfurther comprises a time and wherein the XR experience corresponds tothe time.
 4. The system of claim 3, wherein at least one of the one ormore actions taken on the user device while viewing the XR experiencecomprises a query and at least a portion of the fourth set ofinformation comprises a response to the query.
 5. The system of claim 4,wherein the XR experience is a first XR experience, the query comprisesa hyperlink, and the response to the query is configured to cause thegraphical user interface to display a second XR experience.
 6. Thesystem of claim 5, wherein the second XR experience comprises a virtualenvironment.
 7. The system of claim 6, wherein the first XR experiencecomprises text and images superimposed over a video stream of a physicalenvironment.
 8. The system of claim 1, wherein the XR experiencecomprises a virtual object placed in a real world environment.
 9. Asystem comprising: a server comprising a database, one or more firstprocessors and a first memory comprising first instructions, the servercommunicatively coupled to a user device; wherein the firstinstructions, when executed by the one or more first processors, causethe server to perform operations comprising: receiving, from the userdevice, a first command to create new AR content; displaying, via theuser device, a first plurality of options each corresponding to a typeof AR content; receiving, from the user device, a second commandspecifying a first one of the first plurality of options; generating anAR experience of the type corresponding to the first one of theplurality of options; displaying the AR experience via the user device;and storing the AR experience in the database.
 10. The system of claim9, wherein the first one of the plurality of option corresponds to abase AR experience and the first instructions, when executed by the oneor more first processors, cause the server to perform further operationscomprising: displaying, via the user device, a second plurality ofoptions each corresponding to a type of base AR experience taken fromthe group comprising an image-recognition event, a model placement, aninstruction guide, a virtual award, a virtual video message, and an ARlocation experience; and receiving, from the user device, a thirdcommand specifying a first one of the second plurality of options;wherein the generated AR experience is of the type of base AR experiencecorresponding to the first one of the second plurality of options. 11.The system of claim 10, wherein the generated AR experience comprisesone or more content items received from the user device.
 12. The systemof claim 11, wherein the one or more content items received from theuser device comprise one or more of a photograph, a video, and athree-dimensional model.
 13. The system of claim 12, wherein the one ormore content items received from the user device comprise a real worldlocation.
 14. The system of claim 13, wherein the first instructionsfurther comprise one or more robot process automation scripts and thegenerated AR experience is created at least in part using the one ormore robot process automation scripts.
 15. The system of claim 14,wherein the first instructions further comprise one or more mediaconversion scripts and the first instructions, when executed by the oneor more first processors, cause the server to execute the one or moremedia conversion scripts on the one or more content items to create atleast a portion of the generated AR experience.
 16. The system of claim15, wherein the first instructions, when executed by the one or morefirst processors, cause the server to make a determination as to whethera first one of the one or more content items is a photograph, a video,or a three-dimensional model and select a first one of the one or moremedia conversion scripts based on the determination.
 17. The system ofclaim 16, wherein the generated AR experience is displayed using theuser device.
 18. A method of displaying an XR experience on a userdevice comprising a display, an input device, a first sensor, and asecond sensor, the method comprising the steps of: displaying, on thedisplay of a user device, a plurality of options each corresponding toan XR experience; receiving, from the input device of the user device, aselection corresponding to a first one of the plurality of options;displaying, on the display of the user device, the XR experiencecorresponding to the selection, receiving, from the first sensor, firstsensor data corresponding to a position of the user device whiledisplaying the XR experience; receiving, from the second sensor, secondsensor data corresponding to information regarding a real worldenvironment proximate the user device; receiving, from the input device,third sensor data corresponding to user inputs provided while displayingthe XR experience on the display; and adjusting the displayed XRexperience based at least in part on the first sensor data, the secondsensor data, and the third sensor data.
 19. The method of claim 18,wherein the second sensor comprises a camera and the second sensor datacomprises an image taken from the camera and wherein the method furthercomprises superimposing an AR object in the image while displaying theXR experience.
 20. The method of claim 19, wherein the AR objectcomprises a rendering of a physical object visible in the image.